Electronic acquisition of a hand formed expression and a context of the expression

ABSTRACT

Embodiments include an apparatus, device, method, and computer program product. In an embodiment, a device includes a handheld writing device that includes a writing element, and a writing detector module operable to generate information indicative of a handwriting movement of the writing element. The device also includes a context detector module operable to generate information indicative of a content portion of a document proximate to the handwriting movement of the writing element.

SUMMARY

An embodiment provides a device. The handheld writing device includes awriting element, and a writing detector module operable to generateinformation indicative of a handwriting movement of the writing element.The device also includes a context detector module operable to generateinformation indicative of a content portion of a document proximate tothe handwriting movement of the writing element. The handheld writingdevice may further include a recognition module operable to generateinformation indicative of a human readable content of the hand drivenmovement of the writing element in response to the informationindicative of the hand driven movement of the writing element. Inaddition to the foregoing, other device embodiments are described in theclaims, drawings, and text forming a part of the present application.

Another embodiment provides an electronic pen device. The electronic pendevice includes a generally elongated body, and a writing elementextending from the body. The pen includes a writing detector moduleoperable to generate information indicative of a hand driven movement ofthe writing element. The pen further includes a context detector moduleoperable to generate information indicative of a content portion of adocument proximate to the hand driven movement of the writing element.The electronic pen may also include a communications module. Theelectronic pen may further include a communications module operable totransmit the information indicative of a handwriting movement of awriting element and to transmit the information indicative of at least aportion of a document proximate to the writing movement of the writingelement. The electronic pen may also include a digital storage medium.In addition to the foregoing, other device embodiments are described inthe claims, drawings, and text forming a part of the presentapplication.

A further embodiment provides an apparatus. The apparatus includesoperability to mark a document, and generate an indicator of a positionof the mark relative to an existing character of the document. Theapparatus also includes operability to save the mark and the indicatorwithin the pen. In addition to the foregoing, other apparatusembodiments are described in the claims, drawings, and text forming apart of the present application.

An embodiment provides a device. The device includes a pen configured tomark relative to an existing character of a document based, at least inpart, on a marking position relative to the existing character of thedocument, wherein the mark and an indicator of the marking position aresaved within the pen. In addition to the foregoing, other deviceembodiments are described in the claims, drawings, and text forming apart of the present application.

Another embodiment provides an apparatus. The apparatus includes astylus configured to write on a surface, and a position sensor operableto generate information indicative of handwriting movements of thestylus relative to the surface. The apparatus also includes an imagingelement operable to generate information representing at least a portionof a content displayed from the surface proximate to the handwritingmovements. In addition to the foregoing, other apparatus embodiments aredescribed in the claims, drawings, and text forming a part of thepresent application.

An embodiment provides a method. The method includes generating in ahandheld device a first information indicative of a handwriting movementof a writing element physically associated with the handheld device. Themethod further includes generating in the handheld device a secondinformation indicative of a content portion of a document proximate tothe handwriting movement of the writing element. In addition to theforegoing, other method embodiments are described in the claims,drawings, and text forming a part of the present application.

An embodiment provides a method. The method includes marking at least aportion of a document in response to a hand driven movement of ahandheld marking device, and generating in the handheld marking device afirst data set indicative of the marking. The method further includesgenerating in the handheld marking device a second data set indicativeof a content portion of the document proximate to the marking. Themethod may include saving the first data set indicative of the markingin a digital storage medium physically associated with the handhelddevice. The method may also include saving the second data setindicative of a content portion of the document proximate to the markingin a digital storage medium physically associated with the handhelddevice. The method may further include transmitting from the handheldmarking device the first data set indicative of the marking and thesecond data set indicative of a content portion of the documentproximate to marking. The method may include transmitting the first dataset indicative of the marking from the handheld device. The method mayalso include transmitting the second data set indicative of a contentportion of the document proximate to marking from the handheld device.In addition to the foregoing, other method embodiments are described inthe claims, drawings, and text forming a part of the presentapplication.

An embodiment provides a computer program product. The computer programproduct encodes a computer program for executing a computer process in ahandheld computing device. The computer process includes generating inthe handheld device a first data set indicative of a handwritingmovement of a writing element physically associated with the handhelddevice. The computer process also includes generating in the handhelddevice a second data set indicative of a content portion of a documentproximate to the handwriting movement of the writing element. Thecomputer program product may include saving in a digital storage mediumphysically associated with the handheld device the first data setindicative of a handwriting movement and the second data set indicativeof a content portion of the document proximate to the marking. Thecomputer program process may also include transmitting the first dataset indicative of the marking. The computer program process may alsoinclude transmitting the second data set indicative of a content portionof the document proximate to the marking. In addition to the foregoing,other computer program product embodiments are described in the claims,drawings, and text forming a part of the present application.

In addition to the foregoing, various other embodiments are set forthand described in the text (e.g., claims and/or detailed description)and/or drawings of the present application.

The foregoing is a summary and thus by necessity containssimplifications, generalizations and omissions of detail. Consequently,those skilled in the art will appreciate that the summary isillustrative only and is not intended to be in any way limiting. Otheraspects, inventive features, and advantages of the devices and/orprocesses described herein, as defined solely by the claims, will becomeapparent in the detailed description set forth herein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary system in which embodiments may beimplemented, including a thin computing device;

FIG. 2 illustrates another exemplary system in which embodiments may beimplemented, including a general-purpose computing device;

FIG. 3 illustrates an environment that includes partial view of anexemplary handheld writing device, and an exemplary documentenvironment;

FIG. 4 illustrates a partial view of an environment that includes anexemplary electronic pen, and the exemplary document environment of FIG.3;

FIG. 5 illustrates a partial view of an environment that includes anexemplary apparatus, and the exemplary document environment of FIG. 3;

FIG. 6 illustrates a partial view of an environment that includes anexemplary apparatus, and the exemplary document environment of FIG. 3;

FIG. 7 illustrates an exemplary operational flow;

FIG. 8 illustrates another exemplary operational flow;

FIG. 9 illustrates an alternative embodiment of the exemplaryoperational flow of FIG. 8;

FIG. 10 illustrates another alternative embodiment of the exemplaryoperational flow of FIG. 8; and

FIG. 11 illustrates a partial view of an exemplary computer programproduct that includes a computer program for executing a computerprocess in a handheld computing device.

DETAILED DESCRIPTION

In the following detailed description of exemplary embodiments,reference is made to the accompanying drawings, which form a parthereof. In the several figures, like referenced numerals identify likeelements. The detailed description and the drawings illustrate exemplaryembodiments. Other embodiments may be utilized, and other changes may bemade, without departing from the spirit or scope of the subject matterpresented here. The following detailed description is therefore not tobe taken in a limiting sense, and the scope of the claimed subjectmatter is defined by the appended claims.

Readers often make marks, such as written annotations, on a portion ofprinted materials, such as text and illustrations, using common writinginstruments, such as handheld pens and pencils. Some readers may wish tosave their annotations and the portion of the printed material, orcontext, to which the annotation was directed in electronic form forlater viewing and/or searching. Further, other readers may wish tocreate electronic files that include at least two annotations they madeto a printed material over time, such as their annotations to a favoritebook or seminal work. A writing instrument operable to electronicallyacquire both annotations and a context of the annotations may provide anadvantage to readers.

FIG. 1 and the following discussion are intended to provide a brief,general description of an environment in which embodiments may beimplemented. FIG. 1 illustrates an exemplary system that includes a thincomputing device 20 that interfaces with an electronic device (notshown) that includes one or more functional elements 51. For example,the electronic device may include any item having electrical and/orelectronic components playing a role in a functionality of the item,such as a limited resource computing device, a digital camera, a cellphone, a printer, a refrigerator, a car, and an airplane. The thincomputing device 20 includes a processing unit 21, a system memory 22,and a system bus 23 that couples various system components including thesystem memory 22 to the processing unit 21. The system bus 23 may be anyof several types of bus structures including a memory bus or memorycontroller, a peripheral bus, and a local bus using any of a variety ofbus architectures. The system memory includes read-only memory (ROM) 24and random access memory (RAM) 25. A basic input/output system (BIOS)26, containing the basic routines that help to transfer informationbetween sub-components within the thin computing device 20, such asduring start-up, is stored in the ROM 24. A number of program modulesmay be stored in the ROM 24 and/or RAM 25, including an operating system28, one or more application programs 29, other program modules 30 andprogram data 31.

A user may enter commands and information into the computing device 20through input devices, such as a number of switches and buttons,illustrated as hardware buttons 44, connected to the system via asuitable interface 45. Input devices may further include atouch-sensitive display screen 32 with suitable input detectioncircuitry 33. The output circuitry of the touch-sensitive display 32 isconnected to the system bus 23 via a video driver 37. Other inputdevices may include a microphone 34 connected through a suitable audiointerface 35, and a physical hardware keyboard (not shown). In additionto the display 32, the computing device 20 may include other peripheraloutput devices, such as at least one speaker 38.

Other external input or output devices 39, such as a joystick, game pad,satellite dish, scanner or the like may be connected to the processingunit 21 through a USB port 40 and USB port interface 41, to the systembus 23. Alternatively, the other external input and output devices 39may be connected by other interfaces, such as a parallel port, game portor other port. The computing device 20 may further include or be capableof connecting to a flash card memory (not shown) through an appropriateconnection port (not shown). The computing device 20 may further includeor be capable of connecting with a network through a network port 42 andnetwork interface 43, and through wireless port 46 and correspondingwireless interface 47 may be provided to facilitate communication withother peripheral devices, including other computers, printers, and so on(not shown). It will be appreciated that the various components andconnections shown are exemplary and other components and means ofestablishing communications links may be used.

The computing device 20 may be primarily designed to include a userinterface having a character, key-based, other user data input via thetouch sensitive display 32 using a stylus (not shown). Moreover, theuser interface is not limited to an actual touch-sensitive panelarranged for directly receiving input, but may alternatively or inaddition respond to another input device, such as the microphone 34. Forexample, spoken words may be received at the microphone 34 andrecognized. Alternatively, the computing device 20 may be designed toinclude a user interface having a physical keyboard (not shown).

The device functional elements (not shown) are typically applicationspecific and related to a function of the electronic device. The devicefunctional elements are driven by a device functional element(s)interface 50, which coupled with the system bus 23. A functional elementmay typically perform a single well-defined task with little or no userconfiguration or setup, such as a refrigerator keeping food cold, a cellphone connecting with an appropriate tower and transceiving voice ordata information, and a camera capturing and saving an image.

FIG. 2 illustrates another exemplary system in which embodiments may beimplemented. FIG. 2 illustrates an electronic device that may correspondin whole or part to a general-purpose computing device, shown as acomputing system environment 100. Components of the computing systemenvironment 100 may include, but are not limited to, a computing device110 having a processing unit 120, a system memory 130, and a system bus121 that couples various system components including the system memoryto the processing unit 120. The system bus 121 may be any of severaltypes of bus structures including a memory bus or memory controller, aperipheral bus, and a local bus using any of a variety of busarchitectures. By way of example, and not limitation, such architecturesinclude Industry Standard Architecture (ISA) bus, Micro ChannelArchitecture (MCA) bus, Enhanced ISA (EISA) bus, Video ElectronicsStandards Association (VESA) local bus, and Peripheral ComponentInterconnect (PCI) bus, also known as Mezzanine bus.

The computing system environment 100 typically includes a variety ofcomputer-readable media products. Computer-readable media may includeany media that can be accessed by the computing device 110 and includeboth volatile and nonvolatile media, removable and non-removable media.By way of example, and not of limitation, computer-readable media mayinclude computer storage media and communications media. Computerstorage media includes both volatile and nonvolatile, removable andnon-removable media implemented in any method or technology for storageof information such as computer-readable instructions, data structures,program modules, or other data. Computer storage media include, but arenot limited to, random-access memory (RAM), read-only memory (ROM),electrically erasable programmable read-only memory (EEPROM), flashmemory, or other memory technology, CD-ROM, digital versatile disks(DVD), or other optical disk storage, magnetic cassettes, magnetic tape,magnetic disk storage, or other magnetic storage devices, or any othermedium which can be used to store the desired information and which canbe accessed by the computing device 110. Communications media typicallyembody computer-readable instructions, data structures, program modules,or other data in a modulated data signal such as a carrier wave or othertransport mechanism and include any information delivery media. The term“modulated data signal” means a signal that has one or more of itscharacteristics set or changed in such a manner as to encode informationin the signal. By way of example, and not limitation, communicationsmedia include wired media such as a wired network and a direct-wiredconnection and wireless media such as acoustic, RF, optical, andinfrared media. Combinations of any of the above should also be includedwithin the scope of computer-readable media.

The system memory 130 includes computer storage media in the form ofvolatile and nonvolatile memory such as ROM 131 and RAM 132. A basicinput/output system (BIOS) 133, containing the basic routines that helpto transfer information between elements within the computing device110, such as during start-up, is typically stored in ROM 131. RAM 132typically contains data and program modules that are immediatelyaccessible to or presently being operated on by processing unit 120. Byway of example, and not limitation, FIG. 2 illustrates an operatingsystem 134, application programs 135, other program modules 136, andprogram data 137. Often, the operating system 134 offers services toapplications programs 135 by way of one or more application programminginterfaces (APIs) (not shown). Because the operating system 134incorporates these services, developers of applications programs 135need not redevelop code to use the services. Examples of APIs providedby operating systems such as Microsoft's “WINDOWS” are well known in theart.

The computing device 110 may also include other removable/non-removable,volatile/nonvolatile computer storage media products. By way of exampleonly, FIG. 2 illustrates a non-removable non-volatile memory interface(hard disk interface) 140 that reads from and writes to non-removable,non-volatile magnetic media, a magnetic disk drive 151 that reads fromand writes to a removable, non-volatile magnetic disk 152, and anoptical disk drive 155 that reads from and writes to a removable,non-volatile optical disk 156 such as a CD ROM. Otherremovable/nonremovable, volatile/non-volatile computer storage mediathat can be used in the exemplary operating environment include, but arenot limited to, magnetic tape cassettes, flash memory cards, DVDs,digital videotape, solid state RAM, and solid state ROM. The hard diskdrive 141 is typically connected to the system bus 121 through anon-removable memory interface, such as the interface 140, and magneticdisk drive 151 and optical disk drive 155 are typically connected to thesystem bus 121 by a removable non-volatile memory interface, such asinterface 150.

The drives and their associated computer storage media discussed aboveand illustrated in FIG. 2 provide storage of computer-readableinstructions, data structures, program modules, and other data for thecomputing device 110. In FIG. 2, for example, hard disk drive 141 isillustrated as storing an operating system 144, application programs145, other program modules 146, and program data 147. Note that thesecomponents can either be the same as or different from the operatingsystem 134, application programs 135, other program modules 136, andprogram data 137. The operating system 144, application programs 145,other program modules 146, and program data 147 are given differentnumbers here to illustrate that, at a minimum, they are differentcopies. A user may enter commands and information into the computingdevice 110 through input devices such as a microphone 163, keyboard 162,and pointing device 161, commonly referred to as a mouse, trackball, ortouch pad. Other input devices (not shown) may include a joystick, gamepad, satellite dish, and scanner. These and other input devices areoften connected to the processing unit 120 through a user inputinterface 160 that is coupled to the system bus, but may be connected byother interface and bus structures, such as a parallel port, game port,or a universal serial bus (USB). A monitor 191 or other type of displaydevice is also connected to the system bus 121 via an interface, such asa video interface 190. In addition to the monitor, computers may alsoinclude other peripheral output devices such as speakers 197 and printer196, which may be connected through an output peripheral interface 195.

The computing system environment 100 may operate in a networkedenvironment using logical connections to one or more remote computers,such as a remote computer 180. The remote computer 180 may be a personalcomputer, a server, a router, a network PC, a peer device, or othercommon network node, and typically includes many or all of the elementsdescribed above relative to the computing device 110, although only amemory storage device 181 has been illustrated in FIG. 2. The logicalconnections depicted in FIG. 2 include a local area network (LAN) 171and a wide area network (WAN) 173, but may also include other networkssuch as a personal area network (PAN) (not shown). Such networkingenvironments are commonplace in offices, enterprise-wide computernetworks, intranets, and the Internet.

When used in a LAN networking environment, the computing systemenvironment 100 is connected to the LAN 171 through a network interfaceor adapter 170. When used in a WAN networking environment, the computingdevice 110 typically includes a modem 172 or other means forestablishing communications over the WAN 173, such as the Internet. Themodem 172, which may be internal or external, may be connected to thesystem bus 121 via the user input interface 160, or via anotherappropriate mechanism. In a networked environment, program modulesdepicted relative to the computing device 110, or portions thereof, maybe stored in a remote memory storage device. By way of example, and notlimitation, FIG. 2 illustrates remote application programs 185 asresiding on memory device 181. It will be appreciated that the networkconnections shown are exemplary and other means of establishing acommunications link between the computers may be used.

In the description that follows, certain embodiments may be describedwith reference to acts and symbolic representations of operations thatare performed by one or more computing devices, such as computing device110 of FIG. 2. As such, it will be understood that such acts andoperations, which are at times referred to as being computer-executed,include the manipulation by the processing unit of the computer ofelectrical signals representing data in a structured form. Thismanipulation transforms the data or maintains them at locations in thememory system of the computer, which reconfigures or otherwise altersthe operation of the computer in a manner well understood by thoseskilled in the art. The data structures in which data is maintained arephysical locations of the memory that have particular properties definedby the format of the data. However, while an embodiment is beingdescribed in the foregoing context, it is not meant to be limiting asthose of skill in the art will appreciate that the acts and operationsdescribed hereinafter may also be implemented in hardware.

FIG. 2 illustrates an example of a suitable environment on whichembodiments may be implemented. The computing system environment 100 ofFIG. 2 is an example of a suitable environment and is not intended tosuggest any limitation as to the scope of use or functionality of anembodiment. Neither should the environment be interpreted as having anydependency or requirement relating to any one or combination ofcomponents illustrated in an exemplary operating environment.

Embodiments may be implemented with numerous other general-purpose orspecial-purpose computing devices and computing system environments orconfigurations. Examples of well-known computing systems, environments,and configurations that may be suitable for use with an embodimentinclude, but are not limited to, personal computers, server computers,hand-held or laptop devices, personal digital assistants, multiprocessorsystems, microprocessor-based systems, set top boxes, programmableconsumer electronics, network, minicomputers, mainframe computers, anddistributed computing environments that include any of the above systemsor devices.

Embodiments may be described in a general context of computer-executableinstructions, such as program modules, being executed by a computer.Generally, program modules include routines, programs, objects,components, data structures, etc., that perform particular tasks orimplement particular abstract data types. An embodiment may also bepracticed in distributed computing environments where tasks areperformed by remote processing devices that are linked through acommunications network. In a distributed computing environment, programmodules may be located in both local and remote computer storage mediaincluding memory storage devices.

The following includes a series of illustrations depictingimplementations of processes. For ease of understanding, certainillustrations are organized such that the initial illustrations presentimplementations via an overall “big picture” viewpoint and thereafterthe following illustrations present alternate implementations and/orexpansions of the “big picture” illustrations as either sub-steps oradditional steps building on one or more earlier-presentedillustrations. This style of presentation utilized herein (e.g.,beginning with a presentation of a illustration(s) presenting an overallview and thereafter providing additions to and/or further details insubsequent illustrations) generally allows for a rapid and easyunderstanding of the various process implementations.

FIG. 3 illustrates an environment 200 that includes partial view ofexemplary handheld writing device 210, and an exemplary documentenvironment 280. In an embodiment, the exemplary document environment280 (hereafter “document 280”) may include a single sheet, a part of apage, a single page, a poster, a display, multiple pages, a chapter, avolume, and/or a file. In a further embodiment, the document may includea book, such as a bound volume, a box, such as a retail package, acontainer, a carton, a label, a currency bill or note, and/or check. Inanother embodiment, the document may include an article of clothing, anitem, a structure, a vehicle, and/or an indicator, such as a temperaturegauge or speedometer. In an embodiment, a document includes any thinghaving a human recognizable content.

The document 280 includes a content portion 282. The content portion 282may include a word 286, illustrated as a word “Rhett,” a phrase 284,illustrated as the phrase “Frankly my dear, I don't give a damn,” and/orother printed or native content, schematically illustrated as a textualparagraph 288. In another embodiment, the content portion may include atextual content portion. For example, a textual content portion mayinclude words printed on a page, such as a page of a book, and/or wordsprinted on a cover or a spine of a book. In a further embodiment, thecontent portion may include a human language content, for example, textprinted in roman alphabet characters, math symbols, and/or scriptcharacters such as Asian language characters. In another embodiment, thecontent portion may include a human readable content, and/or graphicalcontent. In an embodiment, the content portion may include a graphicalcontent having a meaning or a significance to a human, such as apicture, a drawing, a figure, a photograph, an illustration, a diagram,a doodle, a decoration, a sketch, and/or a portrait. In anotherembodiment, the content portion may be arbitrary and not recognizable byan individual human reader. In a further embodiment, the content portionmay include any content primarily useful to a human reader. For example,a content portion may include a textual and/or graphical label on aretail article, such as clothing, a retail package, or a carton. Acontent portion may include an address written on shipping label or anaddress written on a side of a building. A content portion may includean odometer reading on a vehicle or a temperature reading on athermometer. In another embodiment, the content portion may include anincidental element not primarily employed by a human reader but thataids in distinguishing the document. For example, an incidental elementmay include a coffee stain, a torn page, and/or a physical attribute ofthe document. In an embodiment, a content portion may include a humanand/or machine recognizable characteristic form or shape, such a pair ofskis, a car, and/or a dress.

In an embodiment, the content portion 282 of the document 280 may bedisplayed on a surface 281, such as a paper surface, a surface of asheet of paper, a surface of a newspaper, a surface of a book, and/or asurface of a poster. In a further embodiment, the surface may include asurface of a box, a surface of a container, a surface of a carton, asurface of a label, a surface of a currency bill or note, and/or asurface of a check. In another embodiment, a surface may include asurface of an article of clothing, a surface of an item, a surface of astructure, a surface of a vehicle, and/or a surface of an indicator. Inan embodiment, a surface may include content located in at least twoplanes. In another embodiment, a surface may include a planar surface.In an embodiment, the content portion 282 may be electronicallydisplayed, such as a text displayed through the surface 281, and/or atext projected on the surface. The surface electronically displaying thecontent portion may include a surface of a computer monitor, a surfaceof a television screen, and/or a surface of an e-paper,

The exemplary handheld writing device 210 includes a writing element220. In an embodiment, the writing element may include an active writingelement, such as an ink cartridge operable to discharge a marking fluidonto a surface of a document. The active writing element may include areplaceable ink cartridge. In another embodiment, the active writingelement may include a pencil lead. In a further embodiment, the writingelement may include a non-marking element, such as non-marking stylus.

The writing device 210 also includes a writing detector module 230operable to generate information indicative of a handwriting movement250 by the writing element 220. The handwriting movement is illustratedas “My favorite movie line.” The writing detector module 230 may beimplemented in any manner, including software, hardware, firmware,and/or a combination thereof. An embodiment of the writing detectormodule 230 may be implemented using reflected light, optical image,sonar, ultrasound, and/or accelerometer detection of a handwritingmovement. The handwriting movement may be formed in response to handmovements. In an alternative embodiment, the handwriting movement may beformed in response to movements of another part of a body, such as bymovement of a user's head while holding the writing device in theirmouth.

In an embodiment, “information” includes data that may be processed,stored, or transmitted by a computing device. In another embodiment,“information” includes a meaning of data as may be interpreted by ormeaningful to people. In an embodiment, “data” includes a piece orpieces of information. Data may include facts, quantities, characters,and/or symbols. Data may be subject to operations by a computer, saved,and transmitted by electrical signals.

In an embodiment, the writing detector module 230 may be implemented byilluminating a surface 281 of the document 280 from the handheld writingdevice 210, and determining the handwriting movement 250 of the writingelement 220 in response to reflected illumination from the surface, suchas more fully described in U.S. Pat. No. 6,741,335, titled OPTICALTRANSLATION MEASUREMENT, issued on May 25, 2004, Kinrot, et al. asinventor; U.S. Pat. No. 6,452,683, titled OPTICAL TRANSLATIONMEASUREMENT, issued on Sep. 17, 2002, Kinrot, et al. as inventor; andU.S. Pat. No. 6,330,057, titled OPTICAL TRANSLATION MEASUREMENT, issuedon Dec. 11, 2001, Lederer, et al. as inventor, each of which isincorporated herein by reference in their entirety for all that theydisclose and teach. In another embodiment, the writing detector modulemay be implemented by capturing images of a pattern in a writing baseand handwriting movements of the pen determined by calculating positionsof the writing element based on the images of the writing base, such asmore fully described in U.S. Pat. No.______ (published as US2004/0179000) titled ELECTRONIC PEN, MOUNTING PART THEREFOR AND METHODOF MAKING THE PEN, filed Jun. 26, 2002, under application Ser. No.10/179,949, Fermgard, et al. as inventor, which is incorporated hereinby reference in its entirety for all that is discloses and teaches. In afurther embodiment, the writing detector module may be implemented usingultrasound to track a handwriting movement of a writing element, such asmore fully described in U.S. Pat. No.______ (published as US2003/0173121) titled DIGITZER PEN filed Mar. 18, 2002, under applicationSer. No. 10/098,390, Zloter, et al. as inventor, which is incorporatedherein by reference in its entirety for all that it discloses andteaches. In an embodiment, the writing detector module may beimplemented using a sensor package that measures velocity, acceleration,and angular acceleration in a handheld writing device, such as morefully described in U.S. Pat. No. ______ (published as US 2004/0260507)titled 3D INPUT APPARATUS AND METHOD THEREOF, filed Jun. 17, 2004 underapplication Ser. No. 10/868,959, Chang, et al. as inventor, which isincorporated herein by reference in its entirety for all that itdiscloses and teaches. In another embodiment, the writing detectormodule may be implemented using an image processing approach thatcaptures and saves segments of a writing movement as a separate image.The final output is a combination of multiple images superimposed oneach other.

In another embodiment, the writing detector module 230 may generateinformation indicative of a handwriting movement 250 by capturing imagesof the handwriting movement, and stitching the images together.

In an embodiment, the writing detector module 230 may include anelectrical circuit 232 operable to generate information indicative of ahandwriting movement of the writing element. In a further embodiment,the writing detector module 230 may include operability to generateinformation indicative of a movement of the writing element generated byan activity of writing by hand. In another embodiment, the writingdetector module may include operability to generate informationindicative of a movement of the writing element in response to a handgenerated activity.

In a further embodiment, the-writing detector module 230 may generateraw handwriting movement information. In another embodiment, the writingdetector module may process the raw handwriting movement information.

The writing device 210 also includes a context detector module 240operable to generate information indicative of a content portion of adocument proximate to the handwriting movement of the writing element.In an embodiment, a content portion of a document proximate to thehandwriting may include a content portion separated from the handwritingmovement of less than approximately one to two inches. In anotherembodiment, a content portion of a document proximate to the handwritingmovement may include a content portion and a handwriting movement bothon a page. In a further embodiment, a content portion of a documentproximate to the handwriting movement may include a content portion on abody that includes the handwriting movement. In an inanimate example,the content portion may be a book title and/or author printed on a bookspine or cover, and the handwriting movement being on a page of thebook. In an animate example, the content portion may include a portionof a human body, such as a skin surface having a landmark, and thehandwriting movement being on the skin surface and proximate to thelandmark, such a surgeon marking incision lines before beginning aprocedure.

The context detector module 240 may be implemented in any manner,including software, hardware, firmware, and/or any combination thereof.In an embodiment, the context detector module 240 may include anelectrical circuit 242 operable to generate information indicative of acontent portion 282 of a document proximate to the handwriting movementof the writing element 220, such as the phrase 284 and/or the word 286.An embodiment includes a context detector module operable to generateinformation indicative of a content portion of a displayed documentproximate to the handwriting movement of the writing element. A furtheralternative embodiment includes a context detector module operable togenerate information indicative of a content portion of anelectronically displayed document proximate to the handwriting movementof the writing element. Another embodiment includes a context detectormodule operable to generate information indicative of a printed contentportion of a document proximate to the handwriting movement of thewriting element. An embodiment includes a context detector moduleoperable to generate information indicative of a printed content portionof a paper-based document proximate to the handwriting movement of thewriting element. A further alternative embodiment includes a contextdetector module operable to generate information indicative of a contentportion of an e-paper document proximate to the handwriting movement ofthe writing element.

In an embodiment, the context detector module 240 may include an imagecapture device 246 having a lens 248. The image capture device 246includes operability to capture at least one image of the contentportion 282 proximate to the handwriting movement 250, such as thephrase 284 and/or the word 286. The image capture device 246 and thelens 248 may have any location with respect to the handheld writingdevice 210 suitable for capturing at least one image of a contentportion proximate to the handwriting movement, including within an outerperiphery of the handheld writing device, and/or outside the outerperiphery. In another embodiment, the image capture device 246 includesoperability to capture image information of the displayed contentportion proximate to the handwriting movement with a resolutionsufficient to render the image information into textural information byan optical character resolution process.

In a further embodiment, the image capture device 246 includesoperability to capture at least one image of a displayed content portionas a writer moves the writing element 220 of the handheld writing device210 toward the document 280. In another embodiment, the image capturedevice includes operability to capture at least one image of a displayedcontent portion as a writer imparts a handwriting movement to thewriting element. In a further embodiment, the image capture deviceincludes operability to capture at least one image of a displayedcontent portion as a writer moves the writing element of the handheldwriting device away from the document. The lens 248 may include a useradjustable orientation allowing a user to optimize image capture.

In an embodiment, the context detector module 240 may include a useractivatable switch (not shown) allowing a user to select when theinformation indicative of a content portion of a document element willbe captured. For example, the image capture device 246 may include auser activatable switch allowing a user to select when an image of acontent portion is obtained. A user may aim the lens 248 at a contentportion and capture an image by activating the switch. The switch mayinclude a hardware switch, a motion sensitive switch, and/or a voiceactivated switch.

In an embodiment, the context detector module 240 may further includeoperability to acquire contextual information from another source, andto use that contextual information to generate information indicative ofa content portion of a document proximate to the handwriting movement ofthe writing element. For example, the handwriting movement may occurproximate to a surface or a device operable to generate contextualinformation indicative of a content portion of a document proximate tothe handwriting movement. The surface may include a touch sensitivescreen coupled to a circuit operable to generate the contextualinformation. Alternatively, the handwriting movement may occur proximateto a surface monitored by an ultrasonic system coupled to a circuitoperable to generate contextual information indicative of a contentportion of a document proximate to the handwriting movement.

In a further embodiment, the context detector module 240 may generateraw information indicative of a content portion of a document proximateto the handwriting movement of the writing element. In anotherembodiment, the context detector module may process the raw contextportion information.

In another embodiment, the context detector module 240 may include anoperability to generate information indicative of a content portion of adocument proximate to the handwriting movement using other technologies.For example, information indicative of a content portion may begenerated using data from a radio frequency identification device (RFID)associated with the document, a global positioning satellite system,and/or other data signal.

In an alternative embodiment, the handheld writing device 210 mayinclude a recognition module 290. The recognition module includes anoperability to generate information indicative of a human readablecontent in response to the information indicative of the hand drivenmovement of the writing element. For example, in an embodiment, therecognition module may determine a presence of human readable content inthe information indicative of the hand driven or handwriting movement250 generated by the writing detector module 230, recognize the humanreadable content as “My favorite movie line,” and generate informationindicative of that content. In another embodiment, the recognitionmodule may determine a presence of human readable content, recognize thehandwriting movements as forming the human readable content as a scriptcharacter, and generate information indicative of that script character.

In an operation of an embodiment, a reader may wish to annotate thedocument 280 with a comment or sketch. The reader holds the handheldwriting device 210 in their hand in a manner similar to a conventionalpen or pencil, and begins handwriting an annotation using the writingelement 220 proximate to a content portion 282. The handheld writingdevice 210 captures a handwriting movement, or hand driven movement, ofthe writing element imparted by the user's hand to the writing device,and through the writing detector module 230, generates informationindicative of the handwriting movement. The handheld writing devicecaptures a context of the handwriting movement through the contextdetector module 240 by capturing at least one image of the contentportion of the document proximate to the handwriting movement. Thecontext may be captured before the handwriting movement, during thehandwriting movement, and/or after the handwriting movement. The contextdetector module generates information indicative of the content portionof the document proximate to the handwriting movement.

FIG. 4 illustrates a partial view of an environment 201 that includes anexemplary electronic pen 211, and the exemplary document environment 280of FIG. 3. The electronic pen 211 includes a generally elongated body217 and the writing element 220 extending from the generally elongatedbody. In an embodiment, the writing element may be configured to make avisible mark. A visible mark may include any mark visible to a humaneye. For example, a visible mark may include an ink mark, and/or apencil mark. A visible mark may include a letter, a character, a symbol,a line, and/or a figure.

The electronic pen 211 also includes a writing detector module 234operable to generate information indicative of a hand driven movement ofthe writing element. In an embodiment, a hand driven movement includes ahand-generated movement. In an alternative embodiment, the writingdetector module includes an operability to generate informationindicative of a hand driven movement of the writing element in contactwith the surface 281 of a document 280. In another alternativeembodiment, the writing detector module includes an operability togenerate information indicative of a hand driven movement of the writingelement in contact with the surface of a paper document. In a furtherembodiment, the writing detector module includes an operability togenerate information indicative of a hand driven movement of the writingelement in contact with a surface displaying an electronic document. Thewriting detector module may be implemented in any manner, includingsoftware, hardware, firmware, and/or a combination thereof. Anembodiment of the writing detector module may be implemented usingreflected light, optical image, sonar, ultrasound, and/or accelerometerdetection of the handwriting movement. In an embodiment, the writingdetector module may be implemented in a manner at least substantiallysimilar to the writing detector module 230 of FIG. 3.

The electronic pen further includes a context detector module 244operable to generate information indicative of a content portion of adocument proximate to the hand driven movement of the writing element.In an alternative embodiment, the context detector module may include anoperability to generate information indicative of a content portion of apage of multiage document proximate to the handwriting movement of thewriting element. In another alternative embodiment, the context detectormodule may include an operability to generate information indicative ofa hand-generated writing movement of the writing element. The contextdetector module 244 may be implemented in any manner, includingsoftware, hardware, firmware, and/or any combination thereof. In anembodiment, the context detector module 244 may be implemented in amanner at least substantially similar to the writing detector module 240of FIG. 3.

In an alternative embodiment, the electronic pen 211 may include atleast one additional module. Additional modules may include acommunications module 260 and a storage module 270. In an embodiment,the communications module 260 includes an operability to transmit atleast one of the signal indicative of a hand driven movement of awriting element and the signal indicative of at least a portion of adocument proximate to the hand driven movement of the writing element.In another embodiment, the communications module includes an operabilityto transmit in real time at least one of the information indicative of ahand driven movement of a writing element and the information indicativeof at least a portion of a document proximate to the hand drivenmovement of the writing element. In a further embodiment, thecommunications module includes operability to transmit at least one ofprocessed information corresponding to the information indicative of ahandwriting movement of a writing element and processed informationcorresponding to the information indicative of at least a portion of adocument proximate to the writing movement of the writing element.

The communications module 260 may transmit a signal. In an optionalembodiment, the communications module both receives and transmitssignals (transceiver). For example and without limitation, “signal”includes a current signal, voltage signal, magnetic signal, or opticalsignal in a format capable of being stored, transferred, combined,compared, or otherwise manipulated. The communications module mayinclude wireless, wired, infrared, optical, and/or other communicationstechniques. In an embodiment, the communications module may includeoperability for communication with a computing device, such as the thincomputing device 20 of FIG. 1, and/or the computing device 110 of FIG.2. The communications module may include an antenna for wirelesscommunication, a connection for wired connection, and/or an optical portfor optical communication.

The digital storage module 270 may include any suitable digital storagemedium. For example, a digital storage medium may include a computerstorage medium. The digital storage module includes operability to saveat least one of the information indicative of a hand driven movement ofa writing element and the information indicative of at least a portionof a document proximate to the hand driven movement of the writingelement. Information may be saved in any form or format, such as a rawdata form, or a processed data form. In conjunction with thecommunications module 260, information may be saved as generated andthen be available for uploading at a later time. For example,information may be stored, batched, and subsequently transmitted. Inanother example, information may be stored, and subsequently transmittedafter the electronic pen 211 is docked.

In operation of an embodiment, the electronic pen 211 acquires ahandwritten annotation and a context of the handwritten annotation in amanner at least substantially similar to the handheld writing device 210of FIG. 3. In addition, the electronic pen 211 may transmit signalsindicative of the handwritten annotation and the context of theannotation using the communication module 260. One or both of thesignals may be transmitted in real time as a reader writes anannotation. In an alternative embodiment, one or both of the signals maybe processed by a component of the electronic pen prior to transmission.In another alternative embodiment, the electronic pen may save one orboth of the signals in the digital storage medium 270. The saved signalmay be transmitted at a later time.

FIG. 5 illustrates a partial view of an environment 202 that includes anexemplary apparatus 212, and the exemplary document environment 280 ofFIG. 3. The apparatus 212 includes an operability to mark the document280, such as the handwriting movement 250, and to generate an electronicindicator of a position 289 of the mark relative to an existingcharacter, such as the word 286 “Rhett” of the document. The indicator289 of a position of the mark relative to an existing word 286 may beexpressed in any manner, including an indication in Cartesiancoordinates, and/or spherical coordinates. In an embodiment, theexisting character of the document includes at least one existing letterdisplayed by the document. In another embodiment, the existing characterof the document includes at least one existing word displayed by thedocument. In a further embodiment, the document includes at least oneexisting image displayed by the document.

In an alternative embodiment, the apparatus 212 includes a penconfigured to mark relative to an existing character, such as the word286, of a document 280 based, at least in part, on a marking position289 relative to the existing character of the document, wherein the markand an indicator of the marking position are saved within the pen.

FIG. 6 illustrates a partial view of an environment 203 that includes anexemplary apparatus 213, and the exemplary document environment 280 ofFIG. 3. The apparatus 213 includes a stylus 221 configured to write on asurface, such as the surface 281 of the document 280. The stylus may beat least similar to the stylus 220 of FIG. 3. The apparatus alsoincludes a position sensor 236 operable to generate informationindicative of handwriting movement 250 of the stylus relative to thesurface. The apparatus includes an imaging element 245 operable togenerate information representing at least a portion of the word 286displayed from the surface proximate to the handwriting movements.

FIG. 7 illustrates an exemplary operational flow 300. After a startoperation, the operational flow 300 moves to a content operation 310. Atthe content operation 310, a first information is generated in ahandheld device indicative of a handwriting movement of a writingelement physically associated with the handheld device. At the operation320, a second information is generated in the handheld deviceinformation indicative of a content portion of a document proximate tothe handwriting movement of the writing element. The operational flow300 then proceeds to an end operation.

FIG. 8 illustrates an exemplary operational flow 340. After a startoperation, the operational flow 340 moves to a writing operation 345. Atthe writing operation 345, at least a portion of a document is marked inresponse to a hand driven movement of a handheld marking device. At acontent operation 350, a first data set indicative of the marking isgenerated in the handheld marking device. At a context operation 355, asecond data set indicative of a content portion of the documentproximate to the marking is generated in the handheld marking device.

FIG. 9 illustrates an alternative embodiment of the exemplaryoperational flow 340 of FIG. 8. FIG. 9 illustrates an embodiment wherethe exemplary operation flow 340 may include at least one additionaloperation 360. The additional operation 360 may include an operation 362and an operation 364. At the operation 362, the first data setindicative of the marking is saved in a digital storage mediumphysically associated with the handheld device. At the operation 364,the second data set indicative of a content portion of the documentproximate to the marking is saved in a digital storage medium physicallyassociated with the handheld device.

FIG. 10 illustrates an alternative embodiment of the exemplaryoperational flow 340 of FIG. 8. FIG. 10 illustrates an embodiment wherethe exemplary operation flow 340 may include at least one additionaloperation 370. The additional operation 370 may include an operation372, an operation 374, and an operation 378. At the operation 372, thefirst data set indicative of the marking and the second data setindicative of a content portion of the document proximate to marking aretransmitted from the handheld marking device. At the operation 374, thefirst data set indicative of the marking is transmitted from thehandheld marking device. The operation 374 may include at least oneadditional operation, such as an operation 376. At the operation 376,processed data corresponding to the first data set indicative of themarking is transmitted from the handheld marking device. At theoperation 378, the second data set indicative of a content portion ofthe document proximate to marking is transmitted from the handheldmarking device. The operation 378 may include at least one additionaloperation, such as an operation 379. At the operation 379, processeddata corresponding to the second data set indicative of a contentportion of the document proximate to marking is transmitted from thehandheld marking device.

FIG. 11 illustrates a partial view of an exemplary computer programproduct 400 that includes a computer program 404 for executing acomputer process in a handheld computing device. An embodiment of theexemplary computer program product 400 may be provided using acomputer-readable medium 402, and includes computer executableinstructions. The computer product 400 encodes the computer program 404for executing on the handheld computing device a computer process. Thecomputer process includes generating in the handheld device a first dataset indicative of a handwriting movement of a writing element physicallyassociated with the handheld device, and generating in the handhelddevice a second data set indicative of a content portion of a documentproximate to the handwriting movement of the writing element. In analternative embodiment, the computer process 404 may further include anadditional process, such as a process 406, a process 408, and a process410. At the process 406, the first data set indicative of a handwritingmovement and the second data set indicative of a content portion of thedocument proximate to the handwriting movement are saved in a digitalstorage medium physically associated with the handheld device. At theprocess 408, the first data set indicative of the handwriting movementis transmitted. At the process 410, the second data set indicative of acontent portion of the document proximate to the handwriting movement istransmitted. The computer-readable medium 402 may include a computerstorage medium, which may be carried by a computer-readable carrier (notshown). The computer-readable medium 402 may include a communicationsmedium (not shown). The computer program product 400 may be implementedin hardware, software, and/or firmware.

An embodiment includes a system. The system includes a context detectingapparatus operable to generate information indicative of a contentportion of a document, and an annotating apparatus operable to generateinformation indicative of a user-created expression associated with thecontent portion of the document. The context detecting apparatusoperable to generate information indicative of a content portion of adocument may include a handheld context detecting apparatus operable togenerate information indicative of a content portion of a document. Thecontext detecting apparatus operable to generate information indicativeof a content portion of a document may include a human wearable contextdetecting apparatus operable to generate information indicative of acontent portion of a document. The human wearable context detectingapparatus operable to generate information indicative of a contentportion of a document may include eye glasses having a context detectingapparatus operable to generate information indicative of a contentportion of a document.

The annotating apparatus operable to generate information indicative ofa user-created expression associated with the content portion of thedocument may include a handheld annotating apparatus operable togenerate information indicative of a user-created expression associatedwith the content portion of the document. The human handheld annotatingapparatus operable to generate information indicative of a user-createdexpression associated with the content portion of the document mayinclude a handheld annotating apparatus operable to generate informationindicative of a verbal user expression associated with the contentportion of the document. The human handheld annotating apparatusoperable to generate information indicative of a user-created expressionassociated with the content portion of the document may include ahandheld annotating apparatus operable to generate informationindicative of a handwritten user expression associated with the contentportion of the document. The human handheld annotating apparatusoperable to generate information indicative of a user-created expressionassociated with the content portion of the document may include ahandheld annotating apparatus operable to generate an informationindicative of a predetermined user expression associated with thecontent portion of the document.

Those having skill in the art will recognize that the state of the arthas progressed to the point where there is little distinction leftbetween hardware and software implementations of aspects of systems; theuse of hardware or software is generally (but not always, in that incertain contexts the choice between hardware and software can becomesignificant) a design choice representing cost vs. efficiency tradeoffs.Those having skill in the art will appreciate that there are variousvehicles by which processes and/or systems and/or other technologiesdescribed herein can be effected (e.g., hardware, software, and/orfirmware), and that the preferred vehicle will vary with the context inwhich the processes and/or systems and/or other technologies aredeployed. For example, if an implementer determines that speed andaccuracy are paramount, the implementer may opt for a mainly hardwareand/or firmware vehicle; alternatively, if flexibility is paramount, theimplementer may opt for a mainly software implementation; or, yet againalternatively, the implementer may opt for some combination of hardware,software, and/or firmware. Hence, there are several possible vehicles bywhich the processes and/or devices and/or other technologies describedherein may be effected, none of which is inherently superior to theother in that any vehicle to be utilized is a choice dependent upon thecontext in which the vehicle will be deployed and the specific concerns(e.g., speed, flexibility, or predictability) of the implementer, any ofwhich may vary. Those skilled in the art will recognize that opticalaspects of implementations will require optically-oriented hardware,software, and or firmware.

The foregoing detailed description has set forth various embodiments ofthe devices and/or processes via the use of block diagrams, flowdiagrams, operation diagrams, flowcharts, illustrations, and/orexamples. Insofar as such block diagrams, operation diagrams,flowcharts, illustrations, and/or examples contain one or more functionsand/or operations, it will be understood by those within the art thateach function and/or operation within such block diagrams, operationdiagrams, flowcharts, illustrations, or examples can be implemented,individually and/or collectively, by a wide range of hardware, software,firmware, or virtually any combination thereof. In one embodiment,several portions of the subject matter described herein may beimplemented via Application Specific Integrated Circuits (ASICs), FieldProgrammable Gate Arrays (FPGAs), digital signal processors (DSPs), orother integrated formats. However, those skilled in the art willrecognize that some aspects of the embodiments disclosed herein, inwhole or in part, can be equivalently implemented in standard integratedcircuits, as one or more computer programs running on one or morecomputers (e.g., as one or more programs running on one or more computersystems), as one or more programs running on one or more processors(e.g., as one or more programs running on one or more microprocessors),as firmware, or as virtually any combination thereof, and that designingthe circuitry and/or writing the code for the software and or firmwarewould be well within the skill of one of skill in the art in light ofthis disclosure. In addition, those skilled in the art will appreciatethat the mechanisms of the subject matter described herein are capableof being distributed as a program product in a variety of forms, andthat an illustrative embodiment of the subject matter described hereinapplies equally regardless of the particular type of signal bearingmedia used to actually carry out the distribution. Examples of a signalbearing media include, but are not limited to, the following: recordabletype media such as floppy disks, hard disk drives, CD ROMs, digitaltape, and computer memory; and transmission type media such as digitaland analog communication links using TDM or IP based communication links(e.g., packet links).

While particular aspects of the present subject matter described hereinhave been shown and described, it will be apparent to those skilled inthe art that, based upon the teachings herein, changes and modificationsmay be made without departing from this subject matter described hereinand its broader aspects and, therefore, the appended claims are toencompass within their scope all such changes and modifications as arewithin the true spirit and scope of this subject matter describedherein. Furthermore, it is to be understood that the invention is solelydefined by the appended claims. It will be understood by those withinthe art that, in general, terms used herein, and especially in theappended claims (e.g., bodies of the appended claims) are generallyintended as “open” terms (e.g., the term “including” should beinterpreted as “including but not limited to,” the term “having” shouldbe interpreted as “having at least,” the term “includes” should beinterpreted as “includes but is not limited to,” etc.). It will befurther understood by those within the art that if a specific number ofan introduced claim recitation is intended, such an intent will beexplicitly recited in the claim, and in the absence of such recitationno such intent is present. For example, as an aid to understanding, thefollowing appended claims may contain usage of the introductory phrases“at least one” and “one or more” to introduce claim recitations.However, the use of such phrases should not be construed to imply thatthe introduction of a claim recitation by the indefinite articles “a” or“an” limits any particular claim containing such introduced claimrecitation to inventions containing only one such recitation, even whenthe same claim includes the introductory phrases “one or more” or “atleast one” and indefinite articles such as “a” or “an” (e.g., “a” and/or“an” should typically be interpreted to mean “at least one” or “one ormore”); the same holds true for the use of definite articles used tointroduce claim recitations. In addition, even if a specific number ofan introduced claim recitation is explicitly recited, those skilled inthe art will recognize that such recitation should typically beinterpreted to mean at least the recited number (e.g., the barerecitation of “two recitations,” without other modifiers, typicallymeans at least two recitations, or two or more recitations).Furthermore, in those instances where a convention analogous to “atleast one of A, B, and C, etc.” is used, in general such a constructionis intended in the sense one having skill in the art would understandthe convention (e.g., “a system having at least one of A, B, and C”would include but not be limited to systems that have A alone, B alone,C alone, A and B together, A and C together, B and C together, and/or A,B, and C together, etc.). In those instances where a conventionanalogous to “at least one of A, B, or C, etc.” is used, in general sucha construction is intended in the sense one having skill in the artwould understand the convention (e.g., “a system having at least one ofA, B, or C” would include but not be limited to systems that have Aalone, B alone, C alone, A and B together, A and C together, B and Ctogether, and/or A, B, and C together, etc.).

The herein described aspects depict different components containedwithin, or connected with, different other components. It is to beunderstood that such depicted architectures are merely exemplary, andthat in fact many other architectures can be implemented which achievethe same functionality. In a conceptual sense, any arrangement ofcomponents to achieve the same functionality is effectively “associated”such that the desired functionality is achieved. Hence, any twocomponents herein combined to achieve a particular functionality can beseen as “associated with” each other such that the desired functionalityis achieved, irrespective of architectures or intermedial components.Likewise, any two components so associated can also be viewed as being“operably connected,” or “operably coupled,” to each other to achievethe desired functionality. Any two components capable of being soassociated can also be viewed as being “operably couplable” to eachother to achieve the desired functionality. Specific examples ofoperably couplable include but are not limited to physically mateableand/or physically interacting components and/or wirelessly interactableand/or wirelessly interacting components.

1-31. (canceled)
 32. An apparatus having operability to: a) mark adocument; b) generate an indicator of a position of the mark relative toan existing character of the document; and c) save the mark and theindicator within the pen.
 33. The apparatus of claim 32, wherein theexisting character of the document includes at least one existing letterdisplayed by the document.
 34. The apparatus of claim 32, wherein theexisting character of the document includes at least one existing worddisplayed by the document.
 35. The apparatus of claim 32, wherein theexisting character of the document includes at least one existing imagedisplayed by the document.
 36. A device comprising a pen configured tomark relative to an existing character of a document based, at least inpart, on a marking position relative to the existing character of thedocument, wherein the mark and an indicator of the marking position aresaved within the pen. 37-61. (canceled)
 62. A method comprising:discharging a marking substance on a surface proximate to a characterdisplayed by the surface; generating information corresponding to a markformed on the surface by the discharged marking substance; generating anindication of a position of the mark formed on the surface relative tothe character displayed by the surface; and saving data corresponding tothe mark and the indicator of a position of the mark formed on thesurface relative to the character displayed by the surface.
 63. Themethod of claim 62, wherein the discharging a marking substance on asurface proximate to a character displayed by the surface furtherincludes: discharging a marking substance on a surface of a documentproximate to a character displayed by the surface.
 64. The method ofclaim 62, wherein the discharging a marking substance on a surfaceproximate to a character displayed by the surface further includes:discharging a marking substance on a surface of an object proximate to acharacter displayed by the surface.
 65. The method of claim 62, whereinthe generating information corresponding to a mark formed on the surfaceby the discharged marking substance further includes: generatinginformation corresponding to at least one of a visible trace, a line, aspot, and/or an impression formed on the surface by the dischargedmarking substance.
 66. A device comprising: means for discharging amarking substance on a surface proximate to a character displayed by thesurface; means for generating information corresponding to a mark formedon the surface by the discharged marking substance; means for generatingan indication of a position of the mark formed on the surface relativeto the character displayed by the surface; and means of saving datacorresponding to the mark and the indicator of a position of the markformed on the surface relative to the character displayed by thesurface.